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DEFINITIONS 

IDA  publishes  the  following  documents  to  report  the  results  ol  its  work. 


Reports 

Reports  are  the  most  authoritative  and  most  carefully  considered  products  IDA  publishes. 
They  normally  embody  results  ol  major  projects  which  (a)  have  a  direct  bearing  on 
decisions  affecting  major  programs,  (bj  address  issues  of  significant  concern  to  the 
Executive  Branch,  the  Congress  and/or  the  public,  or  (c)  address  issues  that  have 
significant  economic  implications.  IDA  Reports  are  reviewed  by  outside  panels  ol  experts 
to  ensure  their  high  quality  and  relevance  to  the  problems  studied,  and  they  are  released 
by  the  President  of  IDA. 

Group  Reports 

Group  Reports  record  the  findings  and  results  of  IDA  established  working  groups  and 
panels  composed  of  senior  individuals  addressing  major  issues  which  otherwise  would  be 
the  subject  of  an  IDA  Report.  IDA  Group  Reports  are  reviewed  by  the  senior  individuals 
responsible  for  the  project  and  others  as  selected  by  IDA  to  ensure  Iheir  high  quality  and 
relevance  to  the  proh|«ms  studied,  and  are  released  by  the  President  of  IDA. 

Papers 

Papers,  also  authoritative  and  carefully  considered  products  ol  IDA.  address  studies  that 
are  narrower  in  scope  than  those  covered  in  Reports.  IDA  Papers  are  reviewed  to  ensure 
that  they  meet  the  high  standards  expected  of  refereed  papers  in  professional  journals  or 
formal  Agency  reports. 

Documents 

IOA  Documents  are  used  for  the  convenience  of  the  sponsors  or  the  analysts  fa)  to  record 
substantive  work  done  in  quick  reaction  studies.  |b)  to  record  the  proceedings  of 
conferences  and  meetings,  (c)  to  make  available  preliminary  and  tentative  results  ol 
analyses,  (d)  to  record  data  developed  in  the  course  of  an  investigation,  or  (e)  to  forward 
information  that  is  essentially  unanalyzed  and  unevaluated.  The  review  ot  IDA  Documents 
is  suited  to  their  content  and  intended  use 
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developed  by  the  Department  of  Defense  (DoD)  that  can  be  made  available  to  civilian 
organizations  to  provide  immediate  support  and  assistance  to  upgrade  skills  for  better 
civilian  employment  opportunities.  The  assessment  focuses  on  interactive  courseware 
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undertaken  by  the  DoD  to  effect  this  transfer,  but  three  sets  of  issues  remain  to  be  iesolved: 
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the  warfighting  missions  of  the  DoD  and  the  instructional  needs  of  non-DoD  activities;  and 
(3)  resources  and  responsibilities  in  non-DoD  activities  must  be  allocated  for  determining 
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SUMMARY 


A.  BACKGROUND 

The  National  Defense  Authorization  Act  for  Fiscal  Year  1991  directed  the  Secretary 
of  Defense  to  provide  an  assessment  of  programs  developed  by  the  Department  of  Defense 
(DoD)  that  can  be  made  available  to  civilian  organizations  to  provide  immediate  support  and 
assistance  to  upgrade  skills  for  better  civilian  employment  opportunities.  This  report  was 
prepared  in  response  to  that  direction. 

B  .  CIVILIAN  REQUIREMENT 

There  are  skill  deficiencies  in  the  American  workforce  that  can  only  be  alleviated  by 
training  or  retraining.  The  skills  that  are  needed  include  both  the  basic  skills  of  reading, 
writing,  mathematics,  speaking,  and  listening  applied  to  the  requirements  of  specific  jobs, 
and  the  technical  skills  needed  to  perform  specific  jobs.  Basic  skills  materials  developed 
for  elementary  and  secondary  schools  cannot  be  used  to  satisfy  the  requirements  for 
training  the  adult  workforce,  and  specific  skill  training  is  often  not  available.  As  a  result, 
an  assessment  needs  to  be  performed  to  determine  the  practical  and  economic  viability  of 
transferring  existing  DoD  instructional  materials  to  the  civilian  sector. 

C.  DOD  INITIATIVES 

For  reasons  of  stability,  effectiveness,  and  cost-effectiveness,  the  DoD  has  relied 
increasingly  on  interactive  courseware  (ICW)  to  meet  its  instructional  requirements.  To 
enhance  the  effectiveness  of  ICW  programs,  several  initiatives  have  been  taken.  These 
include:  (1)  development  of  technical  standards  so  that  ICW  programs  can  be  used  across  a 
variety  of  system  configurations;  (2)  establishment  of  a  DoD  Directive  for  development  and 
management  of  ICW  programs  and  a  database,  the  Defense  Instructional  Technology 
Information  System  (DITIS),  to  facilitate  sharing  ICW  programs  within  and  across  the 
Services;  and  (3)  preliminary  examination  of  the  activities  required  to  make  instructional 
technology  and  instructional  materials  accessible  to  those  outside  of  DoD. 
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D.  THE  TRANSFER  OF  DOD  MATERIALS 


An  examination  of  the  titles  in  the  preliminary  version  of  the  DITIS  database 
found  that  ICW  programs  fall  into  four  categories:  (1)  Literacy  and  General  Education, 
(2)  Technical  Training,  (3)  Managerial  Education,  and  (4)  Professional  Skills  Update  and 
Enhancement.  However,  the  detailed  information  needed  to  determine  transfer  candidacy 
for  any  particular  program  title  is  in  the  process  of  being  gathered  for  entry  into  DITIS,  so 
no  conclusions  can  yet  be  drawn  about  the  number  of  DoD  ICW  programs  that  are 
candidates  for  transfer  to  the  civilian  sector. 

Experience  with  the  Job  Skills  Education  Program  (JSEP)  and  Computer  Assisted 
Medical  Interactive-Video  System  (CaMIS)  ICW  programs  was  briefly  reviewed.  These 
programs  provide  valuable  insights  into  the  amount  of  resourcing  that  may  be  needed 
to  enable  outside  users  to  prepare  DoD  ICW  programs  for  their  own  use.  This  experience 
illustrates  issues  of  "de-greening,”  portability,  and  organizational  mission  and 
responsibility  that  must  be  resolved  before  effective  mechanisms  can  be  established  to 
transfer  any  DoD  ICW  programs  to  outside  users. 

E.  ISSUES 

The  successful  transfer  of  instructional  materials  from  any  agency  to  outside  users 
is  a  complex  undertaking.  To  effect  such  transfer  for  DoD  instructional  materials,  three 
major  types  of  issues  must  be  resolved:  (1)  the  need  to  articulate  specific  civilian 
instructional  requirements;  (2)  the  determination  of  methods  to  overcome  fundamental 
differences  between  materials  based  on  the  warfighting  requirements  of  DoD  and  the 
specific  needs  of  the  civilian  sector;  and  (3)  appropriate  allocation  of  responsibilities  to 
other  Federal  agencies,  such  as  the  Departments  of  Labor,  Commerce,  and  Education,  and 
private  industry,  for  determining  civilian  requirements,  modifying  instructional  content, 
reprogramming  materials  for  different  delivery  systems,  identifying  and  marketing  to 
potential  civilian  users,  and  producing,  warehousing,  and  distributing  materials. 

These  factors  are  especially  costly  and  complex  when  they  involve  technology- 
based  instructional  materials.  These  issues  must  be  resolved  and  adequate  resources  of 
time,  money,  and  personnel  made  available.  It  is  only  in  this  way  that  the  goals  and 
objectives  of  technology  transfer  can  be  satisfied. 
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I.  BACKGROUND 


The  National  Defense  Authorization  Act  for  Fiscal  Year  1991  directed  the  Secretary 
of  Defense  to  provide  an  assessment  of  programs  developed  by  the  Department  of  Defense 
(DoD)  that  can  be  made  available  to  civilian  organizations  to  provide  immediate  support  and 
assistance  to  upgrade  skills  for  better  civilian  employment  opportunities  This  report  was 
prep?\cd  in  response  to  that  direction. 

Interest  in  the  DoD  as  a  source  of  instructional  materials  that  can  be  made  available 
to  outside  activities  has  been  growing  for  several  years.  A  number  of  Congressional 
actions  have  been  taken  to  facilitate  and  govern  transfer  of  DoD  materials  and  technologies 
to  non-DoD  applications.  These  actions  include  the  following: 

(a)  The  Stevenson-Wydler  Technology  Innovation  Act  of  1980.  This  act  focuses 
on  research  and  development.  It  authorizes  the  establishment  of  Centers  for 
Industrial  Technology  and  is  intended  to  stimulate  use  of  federally  funded 
research  and  development  products  by  state  and  local  governments  and  the 
private  sector.  In  the  case  of  instruction,  it  facilitates  and  encourages  transfer 
of  research  and  development  products  from  the  DoD  personnel  research 
laboratories  to  non-DoD  applications. 

(b)  The  Federal  Technology  Transfer  Act  of  1986.  This  act  also  focuses  on 
research  and  development.  It  amends  the  Stevenson-Wydler  Act  by 
authorizing  cooperative  research  and  development  agreements  between  Federal 
and  non-Federal  laboratories  and  by  establishing  a  Federal  Laboratory 
Consortium  for  Technology  Transler  within  the  National  Institute  for 
Standards  and  Technology. 

(c)  The  Training  Technology  Transfer  Act  of  1988.  This  act,  which  is  a  chapter  of 
the  Omnibus  Trade  and  Competitiveness  Act  of  1988,  focuses  on  the  transfer 
of  federally-developed  education  and  training  software  to  non-Federal 
activities.  The  act  is  intended  to  facilitate  transfer  of  education  and  training 
software  from  Federal  agencies  to  applications  that  support  the  education, 
training,  and  retraining  of  industrial  workers,  especially  workers  in  small 
business  concerns.  It  specifically  identifies  the  DoD  as  a  potential  source  of 
instructional  software.  The  act  establishes  an  Office  of  Training  Technology 
Transfer  in  the  Department  of  Education,  which  is  to  maintain  a  clearinghouse 
of  information  on  federally-developed  education  and  training  software. 
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This  report  focuses  on  interactive  courseware  (ICW).  ICW  is  defined  as  computer- 
controlled  courseware  that  relies  on  student  input  to  determine  the  pace,  sequence,  and 
content  of  instruction  delivery.  Courseware  by  itself  refers  to  all  training  materials, 
including  the  curriculum  database  and  all  disks,  tapes,  books,  charts,  and  computer 
programs,  necessary  to  deliver  an  ICW  program. 

The  key  distinction  between  an  ICW  program  and  all  others  is  the  provision, 
through  use  of  computer  technology,  of  interactions  that  tailor  the  instruction  to  the  needs 
of  individual  students.  The  desirability  of  individualization  in  instruction  has  been  noted 
throughout  the  history  of  instruction  (Como  and  Snow,  1986).  By  tailoring  instructional 
content  to  individual  needs,  each  student  receives  the  level  of  detail,  pace,  remediation,  and 
sequence  of  topics  and  interactions  needed  to  learn  the  material  efficiently  within  the  limits 
imposed  by  time  and  access  to  other  instructional  resources.  ICW  programs  provide 
individualized  instruction  within  our  current  group-oriented  instructional  institutions.  This 
advantage,  combined  with  the  fact  that  ICW  programs  tend  to  be  developed  as  stand-alone, 
or  autonomous,  modules  of  instruction,  make  them  promising  candidates  for  smooth 
transfer  to  training  in  the  civilian  sector.  Since  they  are  also  the  concern  of  the  Training 
Technology  Transfer  Act  of  1988,  they  are  the  focus  of  this  report. 

Transfer  of  DoD  education  and  training  materials  and  technologies  is  motivated  by 
civilian  requirements.  These  requirements  are  briefly  reviewed  in  Section  II.  Section  III 
describes  a  number  of  significant  initiatives  taken  by  the  DoD  in  response  to  its  own 
requirements  and  those  of  non-DoD  sectors.  These  initiatives  were  taken  to  improve 
management  of  ICW  programs  within  the  DoD,  but  they  will  also  facilitate  access  and 
transfer  of  DoD  ICW  programs  to  outside  agencies  and  the  private  sector.  Section  IV 
provides  an  assessment  of  the  amount  and  kinds  of  DoD  ICW  programs  that  might  be  made 
available  to  meet  civilian  requirements  and  discusses  some  examples  of  DoD  experience  in 
transferring  ICW  programs  to  outside  agencies.  Section  V  discusses  issues  involved  in 
transferring  ICW  programs. 


2 


II.  CIVILIAN  ENVIRONMENT 


The  interest  in  transferring  DoD  instructional  materials  to  the  civilian  sector  stems 
from  the  continuing  deficit  in  skill  levels  of  workers  and  the  increasing  emphasis  on  the  use 
of  high  technology  to  peiformjobs. 

A.  CIVILIAN  TRAINING  REQUIREMENTS 

An  examination  of  drop  out  rates,  scholastic  achievement  scores,  and  performance 
and  test  data  from  employees  indicates  that  a  substantial  percei  Age  of  persons  in  the  U.S. 
workforce  have  severe  skill  deficiencies.  Deficiencies  in  the  skills  of  persons  in,  or 
entering,  the  U.S.  workforce  have  become  particularly  troublesome  in  the  closing  years  of 
the  twentieth  century  since  greater  demands  are  being  made  on  the  skills  and  competencies 
of  the  American  worker. 

At  the  same  time,  a  recurring  theme  in  many  discussions  about  the  U.S.  economy  is 
that  human  capital  is  becoming  increasingly  important  for  maintaining  economic 
competitb'eness.  A  report  issued  by  the  American  Society  for  Training  and  Development 
(Camevale,  1990)  is  indicative  of  the  tone  of  many  recent  reports  on  this  topic: 

By  all  accounts  the  American  workforce  is  not  ready  to  meet  the  new 
competitive  standards.  It  is  not  adequately  prepared  to  take  advantage  of 
new  technology,  especially  information  technology.  It  has  not  learned  how 
to  achieve  results  through  the  use  of  teams.  For  the  most  part,  it  does  not 
have  sufficient  skills  to  leapfrog  ahead  of  competitors  in  order  to  deliver 
quality,  variety,  convenience,  customization,  and  timeliness.  Business  has 
not  learned  how  to  shift  its  focus  to  the  customer,  how  to  engage  in 
partnerships  with  its  competitors,  or  how  to  think  in  terms  of  global 
markets.  In  short,  it  has  a  great  deal  to  learn  in  a  hurry,  (p.  2-3) 

Carnevale  and  Gainer  (1989)  summarize  a  series  of  analyses  that  found  the 
percentage  of  the  GNP  accounted  for  by  resources  from  the  earth  (such  as  minerals, 
energy,  and  food)  is  declining.  In  1890,  these  resources  accounted  for  50  percent  of  the 
GNP.  Today  these  resources  account  for  10  percent  of  the  GNP,  and  human  resources 
now  account  for  more  than  four- fifths  of  the  nation's  economic  output.  Bishop  (1989) 
analyzed  the  impact  of  training  on  the  productivity  of  workers  and  found  that  workers  who 
receive  formal  job  training  have  a  30  percent  higher  productivity  level  than  workers  who 


are  not  formally  trained.  With  regard  to  the  benefits  of  training  for  workers,  those 
receiving  on-the-job  training  have  a  25  percent  salary  advantage  over  workers  who  do  not 
receive  formal  training  (Camevale  and  Gainer,  1989). 

The  American  Society  for  Training  and  Development  (1990)  has  estimated  that  in 
1989  the  total  cost  to  retrain  the  American  workforce  who  did  not  receive  training  would 
have  been  $64  billion  of  which  $45.4  billion  would  have  been  the  employer  share,  and  the 
remainder,  consisting  mostly  of  tuition  and  related  instructional  fees,  would  have  been 
made  by  the  workers  themselves.  The  number  and  percent  of  workers  who  needed  but  did 
not  receive  retraining,  and  the  cost  of  that  retraining,  are  shown  as  follows: 


Type  of  Retraininq 

Number  &  Percent 

Who  Needed  but  Did  Not 
Receive  Retraininq  in  1989 

Money  Needed  for 
Retraining  (Billions), 
Employer  Share 

New  Technology  Training 

Technical  Workers 

9,255,270 

(8%) 

$8.7 

Skilled  Workers 

6,815,680 

(6%) 

6.5 

Executive  Management  and 
Supervisory  Training 

5,490,374 

(5%) 

9.7 

Customer  Service  Training 

11,394,240 

5.4 

(9%) 

Basic  Skills  Training 

16,559,243 

15.1 

(14%) 

Totals 

49,514,807 

(42%) 

$45.4 

Total  Employment  for  1989: 

117,342,000 

The  problem  of  making  substantial  improvements  in  the  skill  levels  of  the  American 
workforce  is  complex  and  unlikely  to  be  solved  by  any  one  action.  Given  the  magnitude  of 
the  problem,  it  is  not  surprising  that  there  is  interest  in  finding  ways  to  ensure  that  existing 
and  effective  training  materials  have  maximum  impact  and  accessibility. 

B  .  CIVILIAN  SKILL  REQUIREMENTS 

Many  reports  dealing  with  training  needs  for  the  workplace  suggest  that  today's 
workers  need  more  and  better  skills.  The  discussions  surrounding  this  position  are  often 


vague  and  fail  to  provide  concrete  definition  of  the  skills  sought.  The  most  commonly  used 
term  to  describe  this  requirement  is  "basic  skills."  The  term  "basic  skills"  is  generally 
compatible  with  the  way  it  was  used  in  the  1990  Office  of  Technology  Assessment  report 
on  worker  training  (i.e.,  basic  skills  are  reading,  writing,  mathematical  concepts  and 
operations,  and  communication  skills— speaking  and  listening).  Without  exception  these 
skills  are  identified  among  the  critical  deficiencies  in  the  workforce.  Every  report  that  was 
examined  included  these  skills  in  their  discussion  of  basic  skills.  However,  basic  skills 
materials  developed  for  elementary  and  secondary  schools  cannot  be  used  to  satisfy  the 
requirements  for  basic  skills  training  in  the  adult  workforce.  Research  dealing  with 
workplace  reading,  mathematics,  and  writing  in  both  military  and  civilian  settings  indicates 
that  on-the-job  reading  tasks  differ  substantially  from  academic  reading  tasks  (Camevale, 
Gainer,  and  Meltzer,  1990). 

Technical  skills  are  also  increasingly  needed.  One  report  (1988)  issued  jointly  by 
the  Departments  of  Labor,  Education,  and  Commerce  provides  two  examples  of  the  ways 
in  which  jobs  require  increasingly  greater  technical  skills.  Even  though  bank  clerks  and 
couriers  have  the  same  title  today  as  they  did  10  years  ago,  their  jobs  are  now  quite 
different.  Bank  clerks  handle  fewer  standardized  requests,  and  more  of  their  time  is  spent 
analyzing  problems,  working  with  incomplete  information,  identifying  sources  of 
information,  and  penetrating  poor  documentation.  Federal  Express,  which  once  hired  high 
school  dropouts  as  couriers,  now  finds  that  it  must  hire  persons  who  can  understand  the 
new  technology  well  enough  to  make  effective  use  of  it.  Most  analyses  of  the  marketplace 
suggest  that  change  will  continue  to  be  the  norm. 

C.  SECTION  SUMMARY 

Sources  of  economic  health  in  the  United  States  have  shifted  from  natural,  physical 
resources  to  the  nation's  workforce,  which  now  accounts  for  more  than  four-fifths  of  the 
nation's  output.  There  are  skill  deficiencies  in  the  American  workforce.  These  deficiencies 
and  the  costs  to  remove  them  increase  every  year  that  the  people  who  need  training  do  not 
receive  it.  In  1989  the  proportion  of  people  who  needed  training  and  did  not  receive  it  was 
estimated  to  be  42  percent  of  the  total  workforce. 

The  skills  that  are  needed  include  both  the  basic  skills  of  reading,  writing, 
arithmetic,  speaking,  and  listening  as  applied  to  the  requirements  of  specific  jobs,  and  the 
technical  skills  needed  to  perform  them.  Basic  skills  materials  developed  for  elementary 
and  secondary  schools  cannot  be  used  to  satisfy  the  requirements  for  basic  skills  training  in 


5 


ft 


the  adult  workforce.  Requirements  for  technical  skills  training  have  grown  because  of 
the  rapid  introduction  of  technology  into  the  workplace.  Some  jobs  (e.g.,  computer 
technician)  start  out  with  technical  requirements,  but  many  jobs  (e.g.,  bank  clerk)  have 
raised  their  technical  requirements  as  technology  for  performing  them  has  been  introduced. 
The  training  that  is  needed  to  meet  these  requirements  must  therefore  cover  both  basic  skills 

and  specific,  but  widely  needed,  technical  skills. 
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III.  DOD  INITIATIVES 


The  courses  presented  by  the  DoD  range  in  length  from  less  than  a  day  to  more  than 
a  year.  The  format  of  this  instruction  ranges  from  traditional  platform  lectures  using 
textbooks  to  correspondence  courses  to  field  exercises  involving  actual  equipment  and  live 
ammunition.  The  instructional  technology  used  also  spans  the  full  range  and  includes 
lesson  plans,  study  guides,  workbooks,  instructor  guides,  overhead  transparencies, 
textbooks,  workbooks,  programmed  textbooks,  audio  cassettes,  films,  videotapes, 
computer-assisted  instruction,  computer-managed  instruction,  interactive  videodisc 
instruction,  CD-ROM  instruction,  part-task  simulators,  two-dimension  simulators,  high- 
fidelity  simulators,  laser-simulated  weapons  fire,  networks  of  simulators,  and  actual 
equipment. 

Increasingly,  the  DoD  relies  on  interactive,  computer-based  courseware  (ICW)  to 
support  its  instructional  needs.  There  are  several  reasons  for  this: 

( 1 )  Stability.  The  development  of  movable  type  and  books  provided  a  capability  to 
capture  the  content  of  high-quality  instruction  and  deliver  it  inexpensively  to 
many  people.  However,  the  effectiveness  and  efficiency  of  instruction  varies 
if  content  is  presented  using  different  rules  and  procedures.  In  addition  to 
content,  computer  technology  captures  the  interactions  of  high-quality 
instruction  and  makes  them  widely,  inexpensively,  and  reliably  available  to 
many  people.  ICW  programs  cannot  yet  supply  all  the  interactions  of  an 
individual  tutor  working  with  an  individual  student,  but  they  can  capture  a 
significant  portion  of  it.  What  interactions  they  do  provide-along  with  the 
instructional  content-can  then  be  delivered  in  a  stable,  autonomous  fashion  to 
learners,  with  a  minimum  of  the  variability  that  is  characteristic  of  human 
instructors. 

(2)  Effectiveness.  There  have  been  enough  reviews  of  the  effectiveness  of 
computer-based  instruction  that  a  review  of  the  reviews  (Niemiec  and  Walberg, 
1987)  has  been  published.  It  indicates  an  improvement  of  about  14  percentile 
points  in  student  achievement  through  the  substitution  of  computer-based 
instruction  for  more  conventional  approaches.  A  suggestion  of  the 
improvements  to  be  expected  in  the  wider  world  of  ICW  using  multimedia 
approaches  were  provided  by  Fletcher  (1990),  who  found  improvements  of 
about  19  percentile  points  through  the  use  of  interactive  videodisc  instruction. 


(3)  Cost-effectiveness.  One  significant  measure  of  cost  when  students  are  being 
paid  by  the  organization  that  trains  them  is  time  spent  in  training.  A  very  stable 
finding  for  ICW  has  been  that  it  reduces  training  time  by  about  30  percent  over 
more  conventional  means  (Orlansky  and  String,  1979;  Fletcher,  1990).  A 
second  source  of  cost  reductions  through  the  use  of  ICW  involves  its  ability  to 
substitute  simulation  for  actual  equipment  without  adversely  affecting  student 
achievement.  Use  of  simulations  that  may  cost  a  tenth  or  even  a  hundredth  of 
the  actual  equipment  used  military  systems  has  provided  a  powerful  cost 
incentive  for  using  ICW  programs  to  train  operators  and  maintainers  of  military 
systems. 

Several  actions  to  improve  the  management  and  use  of  ICW  programs  within  the 
DoD  have  been  initiated.  These  actions  include  the  development  of  policies,  technical 
standards,  and  a  new  automated  reporting  system  and  data  base  that  will  improve  the 
sharing  of  materials  inside  the  DoD  and  outside,  with  non-DoD  activities.  These  initiatives 
were  undertaken  jointly  and  cooperatively  by  the  OASD(FM&P)  Readiness  and  Training 
Directorate  and  the  OASD(PA)  Defense  Audiovisual  Policy  Office.  They  center  on  three 
areas:  (a)  ICW  portability;  (b)  development  and  management  of  ICW;  (c)  procedures  for 
transferring  ICW  within  the  DoD. 

A .  THE  DOD  PORTABILITY  INITIATIVE 

Over  the  next  several  years,  the  DoD  will  continue  to  invest  in  the  development  of 
interactive  instructional  materials.  To  support  these  materials,  the  military  Services--along 
with  many  other  government  and  civilian  organizations— are  acquiring  a  variety  of 
computers,  videodisc  players,  CD-ROM  devices,  and  associated  interfaces.  The  DoD  has 
initiated  a  significant  effort  to  make  its  courseware  portable  and,  therefore,  more  affordable 
and  accessible  to  users  within  and  across  the  Services.  Without  portability,  courseware 
developed  for  one  system  configuration  must  be  re-programmed  to  operate  on  a  different 
system  configuration.  Substantial  development  and  delivery  costs  can  be  avoided  if  ICW 
programs  are  programmed  from  the  beginning  to  operate  on  a  variety  of  system 
configurations.  Portability  of  this  sort  is  accomplished  through  establishment  and 
observance  of  standard  practices  in  the  development  of  ICW  programs. 

The  DoD,  therefore,  has  taken  the  initiative  and  the  lead  in  working  with  industry  to 
develop  technical  standards  that  allow  ICW  programs  produced  on  one  system 
configuration  to  operate  on  many  others.  This  project,  called  Portable  Courseware 
(PORTCO),  brought  together  the  efforts  and  expertise  of  the  Interactive  Video  Industry 
Association,  the  Navy  Personnel  Research  and  Development  Center,  the  Naval  Training 
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Systems  Center,  the  Institute  for  Defense  Analyses,  the  National  Institute  for  Standards  and 
Technology,  the  Defense  Readiness  and  Training  Directorate,  and  the  Defense  Audiovisual 
Policy  Office.  The  intent  of  this  effort  is  to  ensure  as  much  courseware  portability  as 
possible,  without  increasing  the  incremental  costs  to  produce  courseware  in  accord  with 
these  standard  practices. 

j  The  approach  chosen  by  the  DoD  is  based  on  a  standardized,  virtual  interface.  It 

provides  both  the  portability  needed  for  widespread  Service  use  and  the  flexibility  to  satisfy 
the  various  needs  of  different  users,  developers,  and  vendors.  A  standard  based  on  this 
approach  was  developed  by  a  working  group  of  the  Interactive  Multimedia  Association 
^  which  was  supported  jointly  by  the  DoD  and  industry.  The  resulting  set  of  standard 

practices  is  being  widely  adopted  by  industry  and  it  has  been  incorporated  in  MIL-STD- 
1379D,  "Military  Training  Programs"  (5  December  1990).  Based  on  the  DoD  initiative,  the 
National  Institute  for  Standards  and  Technology  (NIST)  has  adopted  the  initial  standard  as 
a  foundation  for  a  Request  for  Architecture  issued  to  solicit  suggestions  and 
recommendations  from  industry  for  further  development.  The  product  of  this  DoD/NIST 
effort  will  be  offered  for  adoption  as  a  Federal,  national,  and  international  standard. 

Next  steps  in  the  DoD  initiative  are  to  include  digital  audio  in  the  standard,  an 
t  extension  that  is  being  developed  in  cooperation  with  another  industry  association,  the 

Aircraft  Industry  Computer-Based  Training  Committee,  to  develop  a  device  level,  "plug 
and  play,"  capability,  and  to  extend  the  standard  to  operating  system  portability  based  on  an 
open  systems  architecture.  The  DoD  has  sought  cooperative  relationships  with  a  number  of 
»  organizational  bodies  in  government  and  industry  that  are  pursuing  the  same  ends.  The  end 

result  should  promote  much  greater  use  of  a  highly  effective  form  of  instruction,  wider 
competition  among  suppliers,  and  lower  per-unit  costs  of  ICW  programs. 

B  .  ICW  DIRECTIVE 

A  new  DoD  Directive  (DoDD  1322.bb,  "Development  and  Management  of 
Interactive  Courseware  (ICW)  for  Military  Training")  has  been  developed  and  formally 
coordinated  with  the  DoD  Components  and  is  about  to  be  released.  The  directive 
ft  establishes  DoD  policy,  prescribes  procedures,  assigns  responsibilities,  and  establishes 

information  requirements  for  using  ICW  programs  to  train  military  personnel.  The  major 
thrusts  of  the  directive  are  to  establish:  (1)  technical  standards  to  promote  the  portability  of 
ICW  programs  between  multiple  delivery  systems,  (2)  a  database  of  DoD  ICW  programs  to 
ft 
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promote  sharing  of  resources  between  DoD  Components,  and  (3)  procedures  to  improve 
the  life-cycle  management  of  ICW  programs. 

Specifically,  the  directive  sets  five  policies  for  the  development  and  management  of 
DoD  ICW  programs  and  materials.  First,  ICW  programs  are  to  be  designed  to  promote 
portability,  folic  ving  the  standard  DoD  programming  protocols  developed  under  the  DoD 
portability  initiative  and  other  technical  requirements  prescribed  in  MIL-STD-1379. 
Second,  the  directive  minimizes  payment  of  royalties,  recurring  license  or  run-time  fees, 
use  taxes,  or  similar  additional  payments  for  courseware  and  associated  materials 
developed  for  or  by  the  DoD.  Third,  the  directive  establishes  the  Defense  Instructional 
Technology  Information  System  (DITTS)  to  provide  an  inventory  and  maintain  a  catalog  of 
DoD  ICW’  programs  for  use  by  all  DoD  Components.  Fourth,  the  directive  requires  that 
reproduction  master  materials  be  archived  for  the  life  cycle  of  each  ICW  program.  Fifth, 
the  directive  establishes  the  policy  that  the  DoD  Components  ensure  the  availability  of  all 
materials  necessary  to  modify  ICW  programs  throughout  their  life  cycles. 

In  support  of  these  policies,  the  directive  also  provides  specific  procedures.  These 
procedures  fall  into  two  categories:  development  procedures  and  management  procedures. 

The  directive  lists  a  number  of  specific  procedures  that  must  be  followed  in 
developing  ICW  programs  for  or  by  DoD  Components.  To  help  maximize  the 
effectiveness  of  DoD  ICW  programs  and  materials,  the  directive  requires  that  all  ICW 
programs  be  based  on  comprehensive  requirements  and  media  selection  analyses  so  that 
program  development  can  be  properly  prioritized  and  targeted.  In  addition,  the  directive 
requires  that  the  DITIS  inventory  of  existing  ICW  programs  be  reviewed  before  program 
development  or  acquisition  to  determine  whether  an  existing  DoD  program  can  be  acquired 
from  another  Component  or  agency  that  will  adequately  meet  the  needs  of  the  development 
effort  under  consideration. 

In  addition,  the  directive  requires  DoD  Components  to  obtain,  to  the  extent 
authorized  by  the  Federal  Acquisition  Regulations,  the  unlimited  rights  or  Government- 
purpose  license  rights  to  the  courseware,  associated  presentation  programs  necessary  to 
interpret  and  execute  the  courseware,  documentation,  and  associated  training  materials  for 
all  ICW  programs  developed.  This  procedure,  combined  with  the  requirements  for  the  use 
of  the  portability  protocols  and  development  of  logistic  support  packages  will  help  ensure 
the  transferability  and  portability-within  the  DoD-of  ICW  programs  and  materials. 


As  far  as  management  procedures  are  concerned,  the  directive  deals  with  two  major 
issues.  The  first  of  these  is  the  establishment  of  life-cycle  management  activities 
responsible  for  each  ICW  program,  and  the  outlining  of  basic  procedures  for  these 
activities  to  ensure  the  portability  and  availability  of  ICW  programs  and  materials  within 
DoD  Components  and  activities.  Secondly,  the  directive  requires  that  DoD  Components 
ensure  the  life-cycle  availability  of  the  version  of  the  authoring  program,  assembly 
language,  or  higher  order  language  compiler  used  to  develop  the  courseware;  source  code 
for  the  courseware;  concomitant  documentation;  all  associated  software  libraries;  and  all 
other  materials  necessary  and  sufficient  to  modify  the  courseware  through  either  archiving 
or  escrow  procedures  as  appropriate.  In  this  way,  availability  of  all  materials  necessary  to 
modify  or  update  ICW  programs  will  be  ensured  throughout  the  life  of  the  the  ICW 
program. 

Much  of  the  policy  and  procedures  stated  in  the  directive  will  increase  the  ability  of 
the  DoD  to  share  ICW  programs  and  materials.  However,  the  directive— and  its 
concomitant  policies  and  procedures— were  developed  to  increase  the  effectiveness  and 
efficiency  of  military  education  and  training.  The  programs  that  will  be  listed  in  the  DITIS 
inventory  have  been  developed  for  military  education  and  training,  and  it  cannot  be 
assumed  that  most  of  them  will  be  appropriate  for  transfer  to  the  civilian  sector.  For  those 
programs  that  are  appropriate,  the  directive  will  help  ensure  that  the  transfer  can  be 
accomplished  in  a  more  efficient  manner. 

The  impact  of  this  directive  on  DoD  management  of  ICW  programs  will  be 
substantial.  It  encourages  free  and  open  competition  among  vendors  providing  ICW 
programs  to  the  DoD  and  can  be  used  as  an  example  by  all  Federal  agencies  in  making 
these  purchases.  When  fully  operational,  DITIS  will  satisfy  the  information  reporting 
requirements  of  the  Department  of  Education  Clearinghouse  established  by  the  Training 
Technology  Transfer  Act  of  1988.  The  Defense  Training  Performance  and  Data  Center, 
which  will  maintain  the  DITIS  file  has  already  begun  to  assemble  information  for  it. 

Perhaps  the  biggest  advantage  of  the  directive  in  terms  of  the  current  report  and 
transfer  of  materials  is  that  it  considers  many  of  the  issues  restricting  the  transfer  of  ICW 
programs  and  materials  and  presents  ways  to  eliminate  them.  Although  the  directive 
focuses  on  military  training  and  education,  it  represents  a  reasoned  approach  to  minimizing 
the  problems  inherent  in  today's  diverse  technology-based  instruction  and  to  maximizing 
the  efficiency  and  effectiveness  of  training  technology  transfer  between  organizations. 
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C.  TRANSFER  PROCEDURES 


Instructional  technologies  such  as  training  devices,  simulation,  programmed 
instruction,  computer-based  instruction,  computer-managed  instruction,  achievement 
testing,  adaptive  testing,  instructional  networking,  systems  approaches  to  instructional 
development,  artificial  intelligence  and  its  application  to  computer-assisted  instruction,  and 
other  instructionally  relevant  technologies  find  their  roots  in  research  and  development 
originally  supported  by  the  DoD  (Blaiwes  and  Regan,  1986);  Fletcher  and  Rockway,  1986; 
Office  of  Technology  Assessment,  1989).  Transfer  of  these  technologies  from  the  DoD  to 
non-DoD  applications  has  been  successful,  and  it  has  had  a  substantial  impact  on  the 
history  of  instruction  in  this  country.  However,  this  transfer  has  not  been  accomplished 
for  free.  In  nearly  every  case,  non-DoD  applications  had  to  be  funded  and  re-developed  by 
their  sponsoring  organizations  in  order  to  demonstrate  the  utility  of  these  technologies  to 
civilian  activities  and  to  initiate  their  implementation  and  adoption  outside  the  DoD. 

Transfer  of  some  training  materials  has  also  been  accomplished.  The  DoD  (as  well 
as  other  Federal  agencies)  has  provided  many  training  and  education  audiovisual  materials, 
some  of  which  are  interactive,  to  the  general  public,  private  industry,  and  educational 
institutions  through  an  existing  structure  within  the  National  Audiovisual  Center  as  outlined 
in  the  Office  of  Management  and  Budget  Circular  A-114,  "Management  of  Audiovisual 
Activities."  Since  this  mechanism  is  already  in  place,  the  DoD  intends  to  follow  an 
analogous  process  for  making  appropriate  interactive  audiovisual  materials  used  for 
instruction  available  in  accord  with  Public  Law  100-418.  Master  copies  of  interactive 
audiovisual  materials  that  are  intended  for  instruction  and  that  are  suitable  for  release  will  be 
reviewed  by  DoD  counsel  to  ensure  that  no  legal  encumbrances  preclude  their  sale  to  the 
public  and  will  be  subjected  to  a  security  review  process  to  determine  if  they  can  be  cleared 
for  public  release.  Interactive  audiovisual  materials  that  receive  both  a  favorable  legal 
review  and  public  clearance  will  then  be  offered  directly  as  they  were  produced  for  the 
DoD.  The  DoD  can  not  take  responsibility  for  any  costs  associated  with  secondary  user 
desired  content  changes,  editing,  duplication,  shipping,  technical  conversion,  licensing, 
patents,  or  royalties  associated  with  these  cleared  audiovisual  materials. 

These  costs  may  represent  a  substantial  investment.  ICW  programs  are  more 
complex  than  linear  audiovisual  programs  and  require  more  resources  to  classify,  test, 
store,  market,  and  distribute  them  than  do  linear  audiovisual  programs.  Among  the  reasons 
for  additional  resource  requirements  are: 
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•  Classification.  Audiovisual  programs  may  be  classified  by  the  type  of  medium 
used  to  present  them.  ICW  programs  (some  of  which  are  audiovisual 
programs)  are  by  their  nature  multimedia  and,  because  their  instructional 
interactions  are  based  on  the  capabilities  of  computer  technology,  they  may 
employ  a  variety  of  instructional  approaches  that  transcends  the  capabilities  of 
any  medium.  It  is  frequently  noted  in  research  literature  that  classification  of 
ICW  programs  based  on  the  media  they  use  is  misleading  (e.g.,  Clark,  1983). 

•  Testing.  Linear  audiovisual  programs  can  generally  be  depended  on  to 
function  properly  when  delivered;  however,  due  to  lack  of  standardization  at  all 
levels,  which  likely  stems  from  the  newness  of  the  technology,  ICW  programs 
cannot.  They  must  be  tested,  and  this  testing  requires  access  to  a  wide  variety 
of  equipment,  equipment  configurations,  interfaces,  software,  authoring 
systems,  operating  systems,  and  the  technical  expertise  to  use  all  these 
properly. 

•  Marketing.  The  functionality  and  many  of  the  technical  aspects  of  linear 
audiovisual  media  and  programs  are  well  understood  by  many  members  of  the 
public.  The  functionality  and  technical  aspects  of  ICW  programs,  valuable  as 
they  are,  are  generally  poorly  understood  by  both  the  public  and  many 
professionals  in  training  and  education.  Explaining  what  an  ICW  program 
offers  and  why  it  is  important  is  a  major,  difficult  task  in  marketing  it. 

•  Delivery.  Similarly,  linear  audiovisual  programs  can  generally  be  transferred 
successfully  to  new  users  with  little  need  for  technical  interaction  before  or 
after  the  hand-off.  Anyone  who  is  familiar  with  the  need  for  hotlines  and 
dealer  support  required  by  users  of  publicly  sold  software  will  understand  the 
necessity  and  value  of  the  continuing  technical  advice  that  must  be  available  to 
users  of  ICW  programs. 

Nonetheless,  some  DoD  instructional  materials  may  be  suitable  for  transfer  to  non- 
DoD  activities.  The  central  question,  is  how  many?  If  there  are  few,  then  the  attention  paid 
and  resources  allocated  to  transferring  material  out  of  the  DoD  may  not  be  worth  the  effort 
already  expended  on  the  question.  If  there  are  many,  then  more  effort  and  resources 
should  be  provided  to  effect  the  appropriate  transfer.  In  assessing  the  amount  of  material 
suitable  for  transfer  it  is  useful  to  consider  the  process,  analogous  to  the  process  now  used 
for  audiovisual  materials,  that  would  most  likely  be  followed  to  arrive  at  the  list  or  catalog 
of  candidate  transferrable  materials.  This  process  would  involve  at  least  three  "filters"  or 
reviews  through  which  ICW  programs  must  pass:  (1)  a  suitability  review;  (2)  a  security 
review;  and  (3)  a  legal  review. 

•  Suitability  Review.  The  first  question  that  arises  about  DoD  instructional 
materia]  is:  Would  anyone  want  it?  It  is  worth  noting  that  a  substantial  amount 
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of  DoD  instructional  material  is  specifically  focused  on  the  main  business  of 
the  DoD,  which  is  warfighting.  There  is  much  instructional  material 
concerning  combat  and  the  operation,  maintenance,  and  deployment  of  materiel 
required  for  combat.  Some  of  this  material  may  be  of  interest  to  the  foreign 
military  sales  community,  but  the  concern  with  this  report  is  with  transfer  of 
materials  to  non-DoD  uses  in  the  United  States,  and  it  is  assumed  that 
instructional  materials  directly  concerned  with  combat  or  with  systems  that  are 
only  used  for  combat  would  be  excluded  from  further  consideration. 

•  Security  Review.  Classified  material  or  material  that  reveals  business 
proprietary  information  or  trade  or  technology  secrets  cannot  be  transferred  to 
the  public  domain.  Instructional  material  that  could  be  transferred  by  the  DoD 
to  non-DoD  activities  would  have  to  be  material  that  was  reviewed  by  the  DoD 
to  determine  if  it  could  be  made  available  for  public  release  and  unlimited 
distribution  outside  the  DoD. 

•  Legal  Review.  The  most  restrictive  review  for  DoD  instructional  materials  to 
pass  may  turn  out  to  be  the  legal  review.  Here  the  question  is:  Does  the  DoD 
have  the  legal  right  to  provide  instructional  materials  for  copying  and 
distribution  by  others?  In  short,  does  the  DoD  own  it?  Much  of  the 
instructional  material  used  by  the  DoD  has  restrictions  on  its  distribution. 
These  restrictions  are  the  result  of  several  factors:  the  software  used  in  ICW 
programs  may  be  copyrighted;  imagery,  music,  and  sound  effects  used  in 
audiovisual  components  may  be  copyrighted;  and  distribution  rights  may  be 
restricted  to  DoD  users  only.  Clearly,  the  DoD  does  not  have  the  right  to  copy 
and  distribute  these  materials. 

In  other  cases,  the  distribution  rights  for  the  materials  themselves  are  unlimited,  but 
the  means  to  present  them  are  limited.  This  is  especially  likely  to  be  true  of  ICW  programs 
in  which  instructional  content--the  curriculum  data  base  of  items,  scenarios,  graphics, 
photographies,  audio/visual  productions,  and  the  like— is  only  part  of  the  instructional 
package.  The  rules  and  procedures  for  presenting  and  individualizing  the  instructional 
content  are  a  separate  matter.  The  content  may  be  wholly  owned  by  the  DoD  with  free  and 
unrestricted  rights  for  its  distribution,  but  the  rules  and  procedures  for  presenting  it  may 
involve  proprietary  algorithms,  licensing,  royalties,  or  other  limitations  to  the  rights  of  the 
DoD  to  transfer  it  to  non-DoD  users. 

These  legal  restrictions  are  intended  to  be  minimized  by  the  new  ICW  Directive. 
However,  it  cannot  solve  the  problem  entirely.  Also,  distribution  rights  to  ICW  programs 
remain  to  some  extent  uncharted  territory.  An  adequate  body  of  decisions  for  establishing 
legal  precedents  for  the  distribution  of  ICW  programs  to  outside  users  has  yet  to  be 
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accumulated.  Clear  criteria  for  developing  ICW  programs  with  well-understood 
distribution  rights  and  clear  criteria  for  determining  distribution  rights  for  existing  ICW 
programs  have  yet  to  be  developed. 

It  is  also  uncertain  what  legal  liability  issues  may  be  associated  with  transferring 
certain  ICW  materials  to  non-DoD  and  public  users.  Potentially  transferrable  materials  may 
have  to  be  legally  reviewed  in  this  context  as  well. 

The  process,  then,  under  consideration  by  the  DoD  for  transferring  ICW  programs 
to  non-DoD  users  centers  on  these  three  reviews.  Security  review,  although  performed  on 
all  materials  transferred  outside  the  DoD,  may  cost  additional  DoD  resources  due  to  the 
increased  workload  associated  with  the  ICW  materials.  Both  the  suitability  review  and 
legal  review  will  certainly  result  in  new,  nonmission-related  costs  that  the  DoD  will  have  to 
bear  and  that  will  require  additional  resourcing. 

Once  these  reviews  are  completed,  the  instructional  materials  that  emerge  will  be 
listed  in  a  catalog  that  will  be  made  available  to  the  Department  of  Education  Clearinghouse. 
In  addition,  there  are  tasks  that  must  be  performed  by  non-DoD  organizations  to  complete 
the  transfer  of  the  materials  to  civilian  users.  Among  these  are  the  following: 

•  The  catalog  must  be  published  and  disseminated  to  potential  users.  This  task 
includes  maintaining  mailing  lists  and  developing  a  capability  for  marketing  the 
DoD  materials.  The  catalog  must  include  materials  newly  entered,  pre-existing 
materials  that  have  been  updated,  and  materials  whose  distribution  status  has 
changed.  The  catalog  should  be  maintained  in  a  digitally  accessible  format  as 
well  as  in  a  paper  format. 

•  Transferred  materials  must  be  physically  stored  with  all  necessary 
environmental  safeguards.  These  materials  must  be  maintained  in  good 
working  order,  and  updated  and  modified  promptly  by  the  organization 
responsible  for  marketing  them.  Software,  hardware,  and  an  in-house 
technical  capability  must  be  available  in  order  to  provide  this  quality  assurance. 

•  Non-DoD  users  will  also  require  technical  assistance  in  installing  the  DoD 
materials  for  their  own  applications,  and  this  assistance  will  have  to  be 
provided  via  telephone  hot  lines,  technical  bulletins,  user  group  seminars,  and 
the  like. 

Other  tasks  may  well  develop  with  experience.  What  is  needed  is  an  adequately 
resourced  materials  transfer  organization  with  clear  responsibility  to  market,  technically 
support,  and  transfer  DoD  ICW  programs  to  non-DoD  users.  If  the  DoD  materials  and 
other  government-developed  ICW  programs  are  of  sufficient  value,  such  an  organization 
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should  become  self-supporting  after  receiving  start  up  support,  but  adequate  seed  funding 
will  have  to  be  provided  and  an  organizational  "home"  for  the  center  will  have  to  be 
established. 

D.  SECTION  SUMMARY 

For  reasons  of  stability,  effectiveness,  and  cost-effectiveness,  DoD  has  relied 
increasingly  on  interactive  courseware  (ICW),  which  uses  many  forms  of  instructional 
materials  and  technology  to  determine  the  pace,  sequence,  and  content  of  instruction  needed 
by  individual  students. 

The  DoD  has  undertaken  several  initiatives  to  improve  the  management  and  use  of 
its  ICW  programs.  These  initiatives  include:  (1)  development  of  technical  standards  for 
portable  ICW  so  that  ICW  programs  can  be  used  across  a  variety  of  system  configurations; 
(2)  preparation  of  a  DoD  Directive  for  management  and  use  of  ICW  to  ensure  that 
portability  and  other  cost-effective  standards  are  observed  and  to  provide  a  database  (the 
Defense  Instructional  Technology  Information  System,  or  DlflS)  that  will  facilitate  sharing 
ICW  programs  within  and  across  the  Services;  and  (3)  development  of  preliminary  transfer 
procedures  for  making  instructional  technology  and  instructional  materials  accessible  to 
outside  agencies  through  performance  of  a  suitability  review,  a  security  review,  and  a  legal 
review  and  provision  of  a  catalog  that  lists  instructional  programs  that  successfully  pass 
these  reviews  and  are  candidates  for  transfer. 
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IV.  TRANSFER  OF  DOD  MATERIALS 


A.  DITIS  REVIEW 

The  proposed  DITIS  database,  when  fully  operational,  will  be  the  only  DoD 
database  that  wall  satisfy  the  information  reporting  requirements  of  the  Training  Technology 
Transfer  Act  of  1988.  The  current  DITIS  database  contains  a  preliminary  listing  of  some  of 
the  ICW  programs  that  are  now  used  in  the  DoD.  Since  full-scale  development  of  the 
database  has  not  yet  been  started,  the  initial  records  that  have  been  entered  are  not  suitable 
for  a  definitive  empirical  analysis.  Although  many  data  items  have  not  been  completed  in 
many  of  the  data  records,  it  was  possible  to  review  the  program  titles  to  make  an  initial 
assessment  of  the  potential  of  DoD  programs  for  transfer  to  outside  agencies. 

About  45  percent  of  the  ICW  programs  now  listed  in  DITIS  are  implemented  on  the 
PLATO  computer-based  instructional  system,  another  45  percent  are  implemented  on  the 
TICCIT  computer-based  instructional  system,  ar.d  the  remaining  10  percent  are  generally 
implemented  on  various  desktop  microcomputer  systems.  The  implementation  systems  are 
probably  atypical  of  most  DoD  ICW  programs,  but  the  subject  matters  they  cover  appear  to 
be  fairly  representative. 

These  preliminary  DITIS  titles  were  examined  for  possible  non-DoD  uses. 
Programs  that  dealt  only  with  military-specific  equipment  or  procedures,  course 
management  tools  without  accompanying  training  packages,  and  non-training  computer 
programs  and  applications  software  were  specifically  omitted.  Following  this  initial 
screening,  two  criteria  for  applicability  were  used  to  screen  the  remaining  ICW  programs: 
(1)  existence  of  a  similar  job  in  the  civilian  sector;  (2)  existence  of  a  civilian  vocational  area 
that  requires  the  skills  taught. 

These  programs  were  then  divided  into  two  groups:  ( 1 )  generic  programs  that  deal 
with  basic  literacy  or  general  education;  and  (2)  more  specific  programs  that  deal  with  job- 
or  vocation-related  skills.  The  first  of  these  categories  relates  to  Congressional  interest  in 
job  skill  enhancement  programs;  the  second  relates  to  Congressional  interest  in  retraining 
and  skill  enhancement. 
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Several  lower  order  categories  were  found  in  the  ICW  programs  that  deal  with  basic 
literacy  and  general  education.  These  categories  covered  topics  in  general  education, 
including  all  basic  verbal  and  mathematical  literacy  skills,  subject  area-specific  preparation 
for  advanced  education,  learning  skills  (including  learning  strategies  and  reference  skills), 
and  personal  development  skills  necessary  to  function  as  an  adult  in  modem  society  (e.g., 
maintaining  a  checking  account).  More  specifically,  the  programs  fell  into  the  categories 
of:  (1)  College  Level  Entrance  Preparation  (CLEP);  (2)  Chemistry;  (3)  General  Education 
Development  (GED)  Preparation;  (4)  Language  Skills;  (5)  Foreign  Language  Training; 
(6)  Learning  Strategies;  (7)  Mathematics  Skills;  (8)  Personal  Development;  (9)  Reference 
Skills;  and  (10)  Miscellaneous. 

Although  some  of  the  Literacy  and  General  Education  programs  were  develor„o 
along  traditional  educational  lines  emphasizing  literacy  fundamentals  isolated  from  specific 
application,  others  (such  as  those  in  the  Job  Skills  Enhancement  Program)  present  basic 
skills  within  a  job-related  or  practical  context. 

As  far  as  programs  related  to  job-  or  vocation-related  skills  are  concerned,  the 
DITIS  records  fell  into  three  major  categories:  Technical  Training,  Managerial  Education, 
and  Professional  Skills  Update  and  Enhancement. 

The  category  of  Technical  Training  includes  ICW  programs  covering  instruction  in 
the  knowledge,  skills,  and  abilities  necessary  to  perform  non-professional  jobs  (i.e,  jobs 
that  do  not  require  training  in  the  liberal  arts  and/or  sciences  and  advanced  study  in  a 
specialized  field).  More  specifically,  the  programs  fell  into  the  categories  of:  (1)  Basic 
Technical  Skills;  (2)  Safety;  and  (3)  Specific  Vocational  Skills. 

The  category  of  Managerial  Education  includes  ICW  programs  covering 
instruction  in  the  skills  necessary  to  direct  and  administer  groups  of  people  and/or 
processes  within  an  organization.  More  specifically,  the  programs  fell  into  the  categories 
of:  (1)  Administration  and  Personnel;  (2)  Career  and  Personnel  Development;  (3)  Equal 
Employment  Opportunity  and  Affirmative  Action;  (4)  Health  Care  Administration; 
(5)  Instruction  and  Training;  (6)  Leadership  and  Management;  (7)  Motivation;  (8)  Office 
Management;  (9)  Production  Control;  (10)  Time  Management;  and  (11)  Miscellaneous. 

The  category  of  Professional  Skills  Update  and  Enhancement  includes  ICW 
programs  covering  instruction  to  maintain,  update,  and  improve  vocation-specific 
knowledge,  skills,  and  abilities  related  to  specialized  fields  requiring  advanced  education 


and  training.  More  specifically,  the  programs  fell  into  the  categories  of  (1)  Engineering  and 
(2)  Medicine. 

Of  the  DITIS  titles  covered  by  this  analysis,  about  10  percent  were  JSEP  programs, 
most  of  which  were  in  the  category  of  Basic  Skills  Training  and  General  Education,  but 
some  of  which  were  in  the  category  of  Technical  Training.  As  stated  above,  the  JSEP 
programs  were  developed  to  teach  skills  within  a  job  context  rather  than  in  isolation.  No 
data  were  available  concerning  the  development  philosophy  for  the  other  programs. 

Although  at  first  it  appears  that  there  are  numerous  ICW  programs  listed  in  the 
preliminary  DITIS  database  that  are  adaptable  to  training  in  the  civilian  sector,  several 
cautions  should  be  noted.  Each  program  was  sorted  on  the  basis  of  its  title  using  the 
criteria  discussed  above.  Many  of  these  preliminary  DITIS  records  do  not  include  a  list  of 
key  words  and  concepts,  so  no  additional  data  were  available  to  perform  the  son. 
Although  rejection  criteria  determined  the  applicability  of  a  program  to  civilian  training,  the 
decision  was  made  to  include  more  rather  than  fewer  programs  until  more  information  can 
be  obtained. 

Since  the  ICW  programs  were  categorized  on  the  basis  of  titles  and  without  a 
comprehensive  examination  of  the  programs  themselves,  there  are  further  complications  to 
note.  One  of  the  greatest  problems  in  transferring  ICW  programs  to  civilian  applications  is 
that  it  is  often  saturated  with  Service-specific  information-ranging  from  personnel  dressed 
in  uniforms  to  references  in  the  instruction  to  policies,  procedures,  acronyms,  and  the  argot 
of  the  military.  Although  DoD  ICW  programs  may  be  transferred  to  civilian  users,  too 
much  military-specific,  or  "green"  (for  Army  green),  information  may  compromise  the 
effectiveness  and/or  acceptability  of  the  instruction  in  civilian  applications. 

As  discussed  above,  there  may  be  legal  complications  in  transferring  DoD  ICW 
programs  to  the  civilian  sector.  If  the  DoD  does  not  own  the  rights  to  the  programs,  it  can 
not  transfer  them  to  another  DoD  or  Government  agency,  let  alone  to  civilian  users,  without 
copyright  infringement  or  other  violations  of  contractual  agreements. 

The  lack  of  comprehensive  data  for  the  current  analysis  requires  that  the  results  of 
this  assessment  be  treated  as  preliminary.  More  information  is  needed  before 
comprehensive,  empirical  analyses  of  current  DoD  ICW  programs  can  be  completed. 
However,  enough  ICW  programs  appear  applicable  to  the  civilian  sector  in  the  preliminary 
DITIS  database  to  warrant  a  closer  examination  of  the  issue.  Although  a  comprehensive, 
empirical  analysis  is  not  yet  possible,  in  the  future  many  more-and  more  complete-records 


will  be  entered  into  DITIS.  The  initial  findings  of  this  study  warrant  further  examination  of 
the  database  at  that  time. 

B  .  EXAMPLES  OF  TRANSFERRED  MATERIALS 

1 .  Job  Skills  Education  Program  (JSEP) 

JSEP  presents  the  most  extensive  and  most  recent  effort  to  transfer  DoD-developed 
materials  to  non-DoD  applications.  JSEP  was  originally  developed  for  the  Army 
Continuing  Education  System  (ACES)  by  Florida  State  University  and  Ford  Aerospace 
Corporation  under  the  direction  of  the  Army  Research  Institute  for  the  Behavioral  and 
Social  Sciences  (ARI).  It  is  designed  to  assist  individuals  to  improve  the  academic  skills 
required  to  learn  a  specific  occupation  and  to  be  better  prepared  for  employment  or 
subsequent  education  and  training.  It  currently  exists  in  its  original  Army  version  and  in  a 
civilian  version  based  on  the  Army  version. 

The  Army  version  is  an  individualized,  computer-based  instructional  system 
designed  to  improve  soldiers'  abilities  to  learn  their  military  jobs— their  Military 
Occupational  Specialities,  or  MOS.  This  version  operates  on  both  the  PLATO®  and 
MicroTICCIT®  computer  systems.  It  consists  of  about  300  lessons  addressing  about  200 
general  academic  skills  identified  by  an  extensive  analysis  of  94  Army  MOS  in  which  about 
85  percent  of  all  enlisted  soldiers  are  employed.  There  are  three  types  of  lessons  in  JSEP: 
(1)  verbal  lessons  that  teach  reading  and  writing  skills;  (2)  quantitative  lessons  that  teach 
mathematics  skills;  and  (3)  learner  strategies  lessons  to  improve  students'  abilities  to  learn 
and  benefit  from  instruction.  Students  are  guided  through  lessons  selected  for  them  by  a 
computer  program  that  keeps  track  of  their  attendance  and  performance  data  and  provides 
progress  reports  for  students,  instructors,  and  administrators. 

The  civilian  version  of  JSEP  was  developed  under  an  interagency  agreement  jointly 
signed  by  the  Departments  of  Education,  Labor,  and  Defense  to  transfer  technology 
developed  by  the  government  to  civilian  sectors.  JSEP  is  the  first  major  computer-based 
instructional  program  to  be  revised  for  civilian  use.  The  civilian  version  takes  advantage  of 
similarities  between  Army  and  civilian  jobs.  In  preparing  the  civilian  version,  the  original 
military  analyses  of  94  MOS  were  used  to  devise  JSEP  instruction  for  about  20  civilian 
occupations  taken  from  the  Dictionary  of  Occupational  Titles.  About  90  percent  of  the 
Army  materials  were  found  to  be  usable  in  the  civilian  version.  The  civilian  version 
operates  on  the  MicroTICCIT  computer  system,  which  networks  individual  student 
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workstations  with  a  central  host  system  and  uses  the  MS/DOS  operating  system.  This 
version  and  initial  evaluations  of  it  were  documented  by  Philippi  (1989)  and  Fisher, 
Philipi,  Frederick,  Moore,  and  Rebata  (1990)  for  the  National  Alliance  of  Business  and  by 
Dick  and  Branson  (1990)  for  Florida  State  University  and  Ford  Aerospace.  The  results  of 
the  evaluations  found  civilian  JSEP  to  be  instructionally  effective  and  well  accepted  by  the 
students.  The  following  findings  are  more  directly  relevant  to  the  issue  of  transferability: 

(a)  Additional  "de-greening"  may  be  needed.  The  students  indicated  that  residual 
indications  of  JSEP's  Army  origins  were  noticeable  but  not  distracting.  JSEP 
instructors  felt  that  all  indicators  of  the  Army  origins  of  JSEP  should  be 
removed.  "Greeness"  may  affect  civilian  acceptance  of  JSEP,  regardless  of  its 
impact  on  instructional  effectiveness. 

(b)  JSEP  computer  systems  are  relatively  expensive  and  incompatible  with 
commonly  available  computer  systems.  The  DoD  portability  initiative  should 
ameliorate  this  problem  in  future  ICW  programs,  but  for  JSEP  it  remains  a 
stumbling  block.  Proprietary  software  and  hardware  is  currently  required  to 
operate  it. 

(c)  More  technical  support  is  needed  for  both  JSEP  courseware  and  software.  As 
was  noted  above  in  contrast  with  linear  audiovisual  programs,  users  need 
ready  access  to  accurate  and  clear  technical  advice  before,  during,  and  after 
implementation  of  ICW  programs. 

Notably,  copyright  of  JSEP  materials  and  source  code  is  not  a  problem.  Copyright 
for  these  items  was  obtained  by  Florida  State  University  and  then  assigned  to  the  Army. 
(Copyrights  cannot  be  owned  by  the  government,  but  they  can  be  assigned  to  it.) 
However,  the  authoring  system  used  for  JSEP  must  be  present  to  operate  JSEP  materials 
and  it  is  proprietary. 

On  the  basis  of  these  results,  it  appears  that  transfer  of  JSEP  to  civilian  application 
is  desirable,  but  more  complex—and  less  "immediate"— than  many  expected  it  to  be.  Efforts 
by  non-DoD  organizations  continue  to  be  made  to  transfer  JSEP  to  non-DoD  organizations, 
which  currently  include  the  Cumberland/Salem  Job  Training  Consortium  in  Bridgeton,  NJ, 
the  National  Education,  Development,  and  Training  Center  operated  jointly  by  Ford  Motor 
Company  and  the  United  Automobile  Workers  at  Ford's  Cleveland,  Ohio,  Engine  Plant  #2, 
the  Greater  Hartford  Alliance  for  Literacy  at  the  Greater  Hartford  (CT)  Community 
College,  the  Alliance  for  Employee  Growth  and  Development  jointly  sponsored  by 
American  Telephone  and  Telegraph  Company,  Communication  Workers  of  America,  and 
the  International  Brotherhood  of  Electrical  Workers,  and  the  Enhanced  Training 
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Opportunities  Program  jointly  sponsored  by  American  Telephone  and  Telegraph  Company 
and  the  International  Brotherhood  of  Electrical  Workers.  Whether  or  not  the  costs  to 
resolve  the  transfer  issues  represent  significant  savings  over  developing  a  new  JSEP  based 
on  new  occupational  analyses  of  civilian  skills  requirements  or  from  the  occupational 
analyses  perfoimed  in  the  beginning,  remains  an  important,  undecided  issue  that  may  be 
setded  by  these  trial  implementations. 

2 .  Computer  Assisted  Medical  Interactive-Video  System  (CAMIS) 

CAMIS  also  provides  an  example  of  the  issues  that  arise  in  the  transfer  of  DoD- 
developed  programs  to  non-DoD  users.  CAMIS  has  been  under  development  by  the  Naval 
Health  Sciences  Education  and  Training  Command  (NHSETC)  for  more  than  10  years 
(Toth  and  Strub,  1987).  It  uses  interactive  videodisc  technology  to  improve  the  quality  and 
availability  of  Navy  medical  training  through  realistic  tutorial  simulations  of  combat,  mass 
casualty,  and  shipboard  emergencies.  As  of  30  September  1990,  29  CAMIS  productions 
had  been  completed  and  released,  and  9  more  were  in  production.  Titles  include  general 
topics  such  as  Homeostasis,  Musculo-Skeletal  System,  Circulatory  System,  Basic  Medical 
Skills,  Oral  Examination  Assisting,  and  Dental  Emergencies.  About  eight  of  the 
productions  (e.g..  Field  Combat  Casualty  Management;  Preventive  Medicine  in  the  Combat 
Theater:  Regimental  Surgeon)  appear  to  be  closely  tied  to  naval  combat  requirements.  Most 
of  the  remaining  30  productions  could  be  used  in  civilian  medical  instruction. 

Portability  has  been  difficult  with  current  CAMIS  materials.  Several  different 
contractors  developed  the  CAMIS  lessons,  and  they  used  different  technical  approaches  so 
that  different  means  are  needed  to  port  CAMIS  lessons  across  different  suites  of  hardware 
and  different  operating  systems  software.  Efforts  are  underway  to  make  all  CAMIS 
lessons  compatible  with  the  portability  guidelines  in  the  new  ICW  Directive  and  the  recently 
signed  MIL-STD-1379D,  both  of  which  are  products  of  the  DoD  portability  initiative.  This 
is  a  retrofitting  process  that  is  needed  within  DoD  management  of  CAMIS,  but  it  would 
have  to  be  performed  in  any  event  if  non-DoD  organizations  were  to  use  these  materials.  If 
the  retrofitting  and  reprogramming  were  not  necessary  for  DoD  use,  the  considerable  costs 
for  these  activities  would  have  to  come  from  non-DoD  sources. 

The  CAMIS  experience  provides  an  example  of  both  what  is  required  to  effect 
transfer  of  ICW  materials  and  the  importance  of  standard  development  procedures  such  as 
those  established  by  the  DoD  portability  initiatives.  Legal  restrictions  on  ownership  do  not 
seem  to  be  a  significant  problem  for  CAMIS  materials.  The  source  software,  the  videodisc 
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imagery,  and  other  information  held  on  videodiscs  appear  to  be  candidates  for  transfer. 
Legal  issues  with  regard  to  liability,  however,  have  yet  to  be  resolved. 

The  Human  Resources  Research  Organization  (HumRRO)  and  Lunaria,  Inc., 
developed  CAMIS  materials  for  NHSETC  and  were  awarded  a  contract  by  the  Department 
of  Transportation's  Small  Business  Innovative  Research  Program  to  investigate  the 
feasibility  of  transferring  CAMIS  materials  to  non-DoD  activities  (Hargan  and  Rice,  1988). 
The  instructional  goal  in  this  case  was  to  use  CAMIS  interactive  videodisc  instruction  to 
develop  and  sustain  the  skills  of  civilian  emergency  medical  technicians  and  paramedics.  In 
general,  the  CAMIS  materials  were  found  to  be  well  received  by  students  and  instructors 
and  instructionally  effective.  There  were  no  problems  with  the  fact  that  both  the  victims 
and  the  health  care  professionals  shown  in  the  program  were  Navy  personnel.  The  main 
problems  centered  on  differences  between  Navy  procedures  and  equipment  for  dealing  with 
emergencies  and  the  procedures  and  equipment  used  in  the  geographical  area  of  the 
assessment.  These  were  not  DoD  versus  civilian  issues,  but  issues  that  arose  because  there 
are  no  standard  medical  operating  procedures  and  equipment  that  are  widely  used  across  all 
civilian  activities. 

C.  SECTION  SUMMARY 

An  examination  of  the  titles  in  the  preliminary  version  of  the  DITIS  database  found 
ICW  programs  in  the  four  categories  of  Literacy  and  General  Education,  Technical 
Training,  Managerial  Education,  and  Professional  Skills  Update  and  Enhancement.  This 
examination  found  many  titles  for  ICW  programs  that  might  be  candidates  for  transfer  to 
non-DoD  activities.  However,  the  data  elements  for  determining  transfer  candidacy  on  any 
basis  other  than  program  title  are  not  yet  collected  or  entered  in  DITIS,  and  conclusions 
cannot  yet  be  drawn  about  how  many  DoD  ICW  programs  are  transferable. 

Experience  with  the  JSEP  and  CAMIS  ICW  programs  was  briefly  reviewed.  This 
experience  showed  that  the  redesign  costs  for  JSEP  were  high  and  suggest  that  substantial 
resourcing  is  needed  to  enable  non-DoD  users  to  prepare  ICW  programs  for  their  own  use. 
This  experience  further  illustrates  issues  of  "de-greening,"  portability,  and  organizational 
mission  and  responsibility  that  must  be  resolved  before  effective  mechanisms  can  be 
established  to  transfer  any  government-developed  ICW  programs  to  non-DoD  users. 


V.  ISSUES 


The  successful  transfer  of  instructional  materials  from  any  agency  to  outside  users 
is  a  complex  undertaking.  Three  major  factors  need  to  be  considered  in  transferring  DoD 
instructional  materials  to  the  civilian  sector:  (1)  the  needs  of  the  civilian  sector,  (2)  assets 
and  limitations  of  the  DoD  in  meeting  these  needs,  and  (3)  allocation  of  responsibilities  and 
resourcing  for  the  process.  These  factors  and  some  of  the  issues  involved  in  each  are 
discussed  in  the  following  subsections. 

A.  CIVILIAN  NEEDS 

There  are  two  basic  needs  for  training  and  retraining  in  the  civilian  sector- 
enhancement  of  basic  literacy  skills  and  general  educational  level  of  the  workforce,  and 
upgrading  skills  of  both  incumbent  and  incoming  workers  to  help  increase  the  United 
States'  economic  viability  in  the  world  market.  As  discussed  in  Section  II,  there  is 
currently  a  need  in  the  adult  civilian  population  in  general  and  in  the  civilian  workforce  in 
particular  to  eliminate  functional  illiteracy  and,  in  many  cases,  raise  the  general  level  of 
education.  Meeting  these  needs  would  enhance  the  ability  of  potential  workers  to  find 
employment,  and  reduce  the  drain  on  the  national  economy  by  reducing  unemployment. 

In  addition,  not  only  do  workers  need  to  have  a  basic  level  of  skill  in  literacy 
fundamentals,  they  must  also  be  able  to  compete  in  the  changing  workplace  environment 
with  its  increased  emphasis  on  high  technology  for  the  production  of  goods  and  services 
that  are  salable  in  today's  global  marketplace.  Such  retraining  and  skill  upgrading  is  one  of 
the  precursors  to  improving  U.S.  standing  in  the  global  marketplace. 

Both  of  these  are  laudable  goals,  and  it  is  only  natural  to  try  to  use  existing 
instructional  materials  to  meet  them.  However,  although  these  general  needs  are 
recognized,  the  specific  requirements  to  fulfill  them  have  not  yet  been  determined  well 
enough  to  know  whether  or  not  transfer  of  materials  from  the  DoD-or  any  organization-is 
worthwhile.  To  attempt  to  fix  one  problem  (i.e.,  civilian  training  and  retraining  needs) 
without  an  in-depth  assessment  and  careful  planning  will  only  create  another  (i.e.,  the 
costly  acquisition  of  materials  that  are  not  usable  due  to  differing  technologies  and  delivery 
media  or  instructional  content  that  does  not  meet  the  training  requirements  of  the  target 
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organization).  An  assessment  of  the  assets  and  some  of  the  limitations  inherent  in  the 
transfer  of  DoD  instructional  materials  without  such  assessment  and  planning  is  discussed 
in  the  following  section. 

B  .  DOD  ASSETS  AND  LIMITATIONS 

The  instructional  materials  developed  by  and  for  the  DoD  are  based  on  a 
comprehensive  analysis  of  military  training  requirements,  and  are  specifically  tailored  for 
the  defense  and  military  arenas.  As  a  result,  a  substantial  amount  of  this  instructional 
material  is  centered  around  the  DoD  mission— warfighting.  For  example,  much  of  the 
DoD's  instructional  material  concerns  combat  and  the  operation,  maintenance,  and 
deployment  of  materiel  required  for  combat.  Most  of  these  materials  are  not  applicable  in 
the  civilian  sector. 

However,  since  both  the  military  and  the  civilian  workforce  draw  on  the  same 
population  for  personnel,  many  of  the  same  basic  education  and  training  problems  exist  in 
both  arenas.  As  a  result,  the  DoD  has  developed  some  instructional  materials  that  deal  with 
the  problems  of  basic  literacy,  general  education,  general  technical  training,  managerial 
education,  and  professional  skill  upgrade  and  enhancement.  However,  as  stated 
previously,  these  materials  are  based  on  a  comprehensive  front  end  analysis  of  the  specific 
military  training  needs.  This  results  in  instructional  materials  that  are  effective  for  their 
intended  military  purpose,  but  are  of  unknown  utility  and  applicability  to  the  civilian  sector. 
Therefore,  the  transfer  of  materials  is  not  just  a  matter  of  making  available  copies  of  DoD 
instructional  materials  to  potential  users,  outside  transfer  agencies,  or  the  Department  of 
Education  Clearinghouse.  To  maximize  effectiveness  and  efficiency  for  all  organizations 
concerned,  transfer  must  be  predicated  on  a  thorough  understanding  of  both  the  civilian 
training  requirements  and  the  content  of  the  DoD  materials. 

Similarly,  even  those  instructional  materials  that  better  meet  the  needs  of  the  civilian 
sector  may  need  to  be  "de-greened"  (i.e.,  modified  to  eliminate  references  to  the  military 
environment  that  are  inappropriate  to  or  detract  from  the  effectiveness  of  training  in  the 
civilian  sector).  This  can  only  be  determined  on  a  case-by-case  basis,  examining  the 
specific  civilian  requirement  and  the  content  of  the  candidate  DoD  program. 

In  addition,  many  of  the  materials-particularly  ICW  materials-will  need  to  be 
designed  for  other  purposes  and  otherwise  modified  for  use  in  civilian  environments.  Such 
modifications  may  include  "de-greening"  and  reprogramming  for  compatibility  with 
different  software  operating  systems  and  delivery  hardware.  Such  redesign  may  not 
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always  be  the  most  cost-effective  means  of  meeting  the  civilian  requirement,  since  these 
modifications  may  frequently  approximate  or  even  exceed  the  cost  of  developing  new 
materials  tailored  to  the  specific  civilian  requirement. 

In  addition  to  the  question  of  instructional  content,  one  must  also  consider  the  legal 
ramifications  of  transferring  DoD  instructional  materials  to  the  civilian  sector.  One  of  the 
most  obvious  of  these  issues  concerns  copyright.  The  DoD  does  not  hold  the  copyright  to 
many  of  the  current  DoD  instructional  materials-and  ICW  programs  and  materials  in 
particular.  In  such  cases,  the  DoD  does  not  have  the  legal  right  to  transfer  the  product  to 
any  agency  other  than  the  one  for  which  it  was  originally  designed.  In  other  cases,  these 
materials  may  be  transferable,  but  only  at  substantial  additional  cost  to  the  outside  user. 

Another  complicating  factor  is  the  issue  of  security  review  of  the  instructional 
materials.  Obviously  any  materials  that  are  classified  or  cannot  pass  the  other  DoD  security 
review  criteria  are  not  candidates  for  transfer.  In  addition,  as  with  the  discussion  of  "de¬ 
greening,"  above,  a  review  of  the  candidate  material  would  have  to  be  made  to  screen  for 
any  content  that  would  have  national  security  implications,  since  once  the  material  is 
transferred  to  the  civilian  sector,  there  are  no  controls  over  its  distribution. 

Another  interesting  legal  complication  to  the  transfer  of  instructional  materials  is  the 
issue  of  liability.  For  example,  medical  training  programs  such  as  those  in  CAMIS  have 
already  raised  questions  concerning  the  limits  of  the  DoD's  liability  regarding  potential 
health  complications  that  could  be  attributed  to  the  civilian  use  of  DoD  medical  procedures. 

Finally,  there  currently  is  great  disparity  among  the  types  of  hardware,  software, 
and  delivery  systems  that  are  available  to  play  ICW  programs.  At  this  time,  it  cannot  be 
assumed  that  a  program  that  was  developed  to  play  on  one  system  will  automatically  play 
on  another  without  expensive  modifications.  As  discussed  in  Section  III,  above,  the  DoD 
has  an  on-going  initiative  to  rectify  this  problem  for  the  military  sector.  However,  not  only 
is  this  initiative  still  in  progress,  but  even  when  fully  implemented  it  will  not  affect  the  same 
problem  in  the  civilian  sector.  As  a  result,  for  many  cases,  the  civilian  recipient  of  the 
program  will  have  to  bear  the  cost  of  reprogramming  the  material  where  possible  so  that  it 
is  usable.  This  may  often  be  more  expensive  than  developing  entirely  new  training 
materials  tailored  to  specific  civilian  needs. 
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C.  RESPONSIBILITIES  AND  RESOURCING 


Despite  the  issues  raised  in  the  previous  two  subsections,  the  DoD  recognizes  that 
there  may  be  training  materials  that  can  transferred  to  the  civilian  sector  such  that  training 
value  is  not  offset  by  incurred  cost.  Moreover,  once  the  DITIS  becomes  fully  operational, 
the  DoD  will  be  in  a  position  to  more  efficiently  identify  ICW  programs  that  may  potentially 
meet  civilian  training  requirements  and  whose  transfer  is  not  limited  by  the  issues  raised 
above. 

However,  not  only  must  the  potential  impediments  to  transfer  discussed  above  be 
resolved,  but  ambiguities  in  the  allocation  of  responsibilities  to  affected  agencies  and  the 
lack  of  sufficient  resources  must  also  be  resolved  before  the  transfer  of  DoD  instructional 
materials  can  become  a  pratical  reality.  These  issues  are  discussed  below. 

Before  further  DoD  actions  are  undertaken,  there  are  many  other  issues  that  must  be 
resolved  to  effect  efficient  and  effective  transfer  of  materials.  These  activities  concern  the 
resourcing,  roles,  responsibilities,  and  inter-relationships  of  the  involved  agencies: 

•  Determination  of  specific  civilian  requirements; 

•  Redesign  and  modification  of  instructional  content; 

•  Reprogramming  to  meet  recipient's  delivery  system  needs; 

•  Identifying  and  marketing  to  potential  civilian  users; 

•  Production,  warehousing,  and  distribution  of  materials. 

These  functions  are  especially  costly  and  complex  when  they  involve  technology- 
based  instructional  materials.  Responsibilities  for  the  above  functions  must  be  clearly 
defined  and  appropriately  allocated  to  the  receiving  organizations,  including  the 
Departments  of  Labor,  Commerce,  and  Education  and  private  industry. 

Finally,  the  issues  discussed  in  this  section  must  be  adequately  resourced  in  terms 
of  time,  money,  and  personnel.  It  is  only  in  this  way  that  the  goals  and  objectives  of 
technology  transfer  can  be  satisfied. 
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